Diffusion models have quickly become the go-to paradigm for generative modelling of perceptual signals (such as images and sound) through iterative refinement. Their success hinges on the fact that the underlying physical phenomena are continuous. For inherently discrete and categorical data such as language, various diffusion-inspired alternatives have been proposed. However, the continuous nature of diffusion models conveys many benefits, and in this work we endeavour to preserve it. We propose CDCD, a framework for modelling categorical data with diffusion models that are continuous both in time and input space. We demonstrate its efficacy on several language modelling tasks.

Can continuous diffusion models bring the same performance breakthrough on natural language they did for image generation? To circumvent the discrete nature of text data, we can simply project tokens in a continuous space of embeddings, as is standard in language modeling. We propose Self-conditioned Embedding Diffusion, a continuous diffusion mechanism that operates on token embeddings and allows to learn flexible and scalable diffusion models for both conditional and unconditional text generation. Through qualitative and quantitative evaluation, we show that our text diffusion models generate samples comparable with those produced by standard autoregressive language models - while being in theory more efficient on accelerator hardware at inference time. Our work paves the way for scaling up diffusion models for text, similarly to autoregressive models, and for improving performance with recent refinements to continuous diffusion.

神经算法推理的基石是解决算法任务的能力，尤其是以一种概括分布的方式。尽管近年来，该领域的方法学改进激增，但它们主要集中在建立专家模型上。专业模型能够学习仅执行一种算法或具有相同控制流骨干的算法的集合。相反，在这里，我们专注于构建通才神经算法学习者 - 单个图形神经网络处理器，能够学习执行各种算法，例如分类，搜索，动态编程，路径触发和几何学。我们利用CLRS基准来凭经验表明，就像在感知领域的最新成功一样，通才算法学习者可以通过“合并”知识来构建。也就是说，只要我们能够在单任务制度中学习很好地执行它们，就可以以多任务的方式有效地学习算法。在此激励的基础上，我们为CLR提供了一系列改进，对CLR的输入表示，培训制度和处理器体系结构，将平均单任务性能提高了20％以上。然后，我们进行了多任务学习者的彻底消融，以利用这些改进。我们的结果表明，一位通才学习者有效地结合了专家模型所捕获的知识。

Top-performing deep architectures are trained on massive amounts of labeled data. In the absence of labeled data for a certain task, domain adaptation often provides an attractive option given that labeled data of similar nature but from a different domain (e.g. synthetic images) are available. Here, we propose a new approach to domain adaptation in deep architectures that can be trained on large amount of labeled data from the source domain and large amount of unlabeled data from the target domain (no labeled targetdomain data is necessary).As the training progresses, the approach promotes the emergence of "deep" features that are (i) discriminative for the main learning task on the source domain and (ii) invariant with respect to the shift between the domains. We show that this adaptation behaviour can be achieved in almost any feed-forward model by augmenting it with few standard layers and a simple new gradient reversal layer. The resulting augmented architecture can be trained using standard backpropagation.Overall, the approach can be implemented with little effort using any of the deep-learning packages. The method performs very well in a series of image classification experiments, achieving adaptation effect in the presence of big domain shifts and outperforming previous state-ofthe-art on Office datasets.

We present a novel dataset named as HPointLoc, specially designed for exploring capabilities of visual place recognition in indoor environment and loop detection in simultaneous localization and mapping. The loop detection sub-task is especially relevant when a robot with an on-board RGB-D camera can drive past the same place (``Point") at different angles. The dataset is based on the popular Habitat simulator, in which it is possible to generate photorealistic indoor scenes using both own sensor data and open datasets, such as Matterport3D. To study the main stages of solving the place recognition problem on the HPointLoc dataset, we proposed a new modular approach named as PNTR. It first performs an image retrieval with the Patch-NetVLAD method, then extracts keypoints and matches them using R2D2, LoFTR or SuperPoint with SuperGlue, and finally performs a camera pose optimization step with TEASER++. Such a solution to the place recognition problem has not been previously studied in existing publications. The PNTR approach has shown the best quality metrics on the HPointLoc dataset and has a high potential for real use in localization systems for unmanned vehicles. The proposed dataset and framework are publicly available: https://github.com/metra4ok/HPointLoc.

近年来，研究人员创建并引入了大量各种代码生成模型。由于对每个新模型版本的人类评估都是不可行的，因此社区采用了自动评估指标，例如BLEU来近似人类判断的结果。这些指标源自机器翻译域，目前尚不清楚它们是否适用于代码生成任务，以及他们与人类对此任务的评估有多一致。还有两个指标，即Codebleu和Ruby，它们是为了估计代码的相似性并考虑了代码属性的。但是，对于这些指标，几乎没有关于他们与人类评估一致的研究。尽管如此，公制得分的最小差异仍用于声称某些代码生成模型的优越性。在本文中，我们介绍了一项有关六个指标的适用性的研究-Bleu，Rouge-L，Meteor，Chrf，Codebleu，Ruby-用于评估代码生成模型。我们对两个不同的代码生成数据集进行了一项研究，并使用人类注释来评估这些数据集上运行的所有模型的质量。结果表明，对于Python单线的Conala数据集，如果模型得分的差异小于5分，则没有一个指标可以正确模拟人类判断，而$ 95 \％$确定性，则使用$> 95 \％$确定性。对于由特定结构类别组成的炉石传说数据集，至少2分的模型得分差异足以声称一种模型比另一个模型的优越性。使用我们的发现，我们得出了有关使用指标来估计代码生成任务的模型性能的几项建议。

在这项工作中，我们提出了一种从IDE中从用户那里收集完成使用日志的方法，并使用它们来训练基于机器学习的模型来排名完成​​候选。我们开发了一组描述候选人及其上下文的功能，并在基于Intellij的IDE的早期访问程序中部署了其匿名集合。我们使用日志从用户那里收集代码完成数据集，并使用它来训练排名catboost模型。然后，我们在两种设置中对其进行了评估：在收集到的完成的一组持有的集合中，并在IDE中的两个不同组的用户对单独的A/B测试中进行了评估。我们的评估表明，使用对过去用户行为日志训练的简单排名模型可显着改善代码完成体验。与默认的基于启发式的排名相比，我们的模型表明，在2.073中执行IDE完成所需的打字操作数量减少到1.832。该方法遵守隐私要求和法律约束，因为它不需要收集个人信息，在客户方面执行所有必要的匿名化。重要的是，它可以连续改进：实施新功能，收集新数据并评估新模型 - 这样，我们自2020年底以来就一直在生产中使用它。

拉曼光谱与机器学习的组合在临床环境中的应用具有重要的希望，作为一种快速，敏感和无标签的识别方法。这些方法在分类数据中表现良好，该数据包含在训练阶段期间发生的类。但是，在实践中，总是存在频谱尚未被采取或尚未知道的物质，并且当输入数据远离训练集并且包括在训练阶段未见的新类，大量的错误记录阳性，这限制了这些算法的临床相关性。在这里，我们表明这些障碍可以通过实现最近推出的熵开路和对象圈丢失功能来克服。为了证明这种方法的效率，我们编制了40种化学类别的拉曼光谱数据库，将它们分成20种生物学相关的氨基酸，10个与生物相关化学品组成的10个不相关的类，以及神经网络没有的10个类别。以前看过，由各种其他化学品组成。我们表明这种方法使网络能够有效地识别未知类，同时在已知的那些对高精度保持高精度，大大减少了误报的数量，同时在已知类上保持高精度，这将允许这种技术弥合实验室之间的差距实验和临床应用。